MathSymb performance measurements
overview
want to understand this report?
mathSimplify (False)/ExprAF2
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 9.63079543431077e-6 | 9.662389682250276e-6 | 9.718287346185578e-6 |
| Standard deviation | 7.766011259604972e-8 | 1.0421047032483039e-7 | 1.3430192168557048e-7 |
Outlying measurements have slight (5.814766303436481e-2%) effect on estimated standard deviation.
mathSimplify (False)/ExprAF3
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.799871507313629e-5 | 2.8047956004934278e-5 | 2.8108336942010625e-5 |
| Standard deviation | 1.1237181959609385e-7 | 1.4546040742198597e-7 | 1.8135862925909892e-7 |
Outlying measurements have no (7.999479708636821e-3%) effect on estimated standard deviation.
mathSimplify (False)/ExprAF4
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.904727445746504e-5 | 2.9209717583915277e-5 | 2.9695834272324173e-5 |
| Standard deviation | 2.958073134546071e-7 | 6.092348198613658e-7 | 1.0213781228446207e-6 |
Outlying measurements have moderate (0.16221976712299813%) effect on estimated standard deviation.
mathSimplify (False)/ExprAF5
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 9.403099985584427e-5 | 9.429114504703261e-5 | 9.449897306691637e-5 |
| Standard deviation | 6.5886121049399e-7 | 7.482445673902692e-7 | 9.252196072865846e-7 |
Outlying measurements have no (9.900000000000023e-3%) effect on estimated standard deviation.
mathSimplify (False)/ExprAF6
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.1555975073264636e-4 | 1.1583327070753244e-4 | 1.1624251831455583e-4 |
| Standard deviation | 6.258649441733688e-7 | 7.538950464241004e-7 | 9.030707338445274e-7 |
Outlying measurements have slight (1.0415512465373922e-2%) effect on estimated standard deviation.
mathSimplify (False)/ExprAF7
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.1325486614670987e-4 | 2.138609399656499e-4 | 2.1435314717470206e-4 |
| Standard deviation | 1.056074385461648e-6 | 1.4214781455444396e-6 | 1.8622428537889727e-6 |
Outlying measurements have slight (1.1903033822035274e-2%) effect on estimated standard deviation.
mathSimplify (False)/ExprAF8
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.6816829296115766e-4 | 3.688316156699756e-4 | 3.6937351807170304e-4 |
| Standard deviation | 1.1971389386291335e-6 | 1.4766794508076895e-6 | 1.908258915349814e-6 |
Outlying measurements have slight (1.3695987654320833e-2%) effect on estimated standard deviation.
mathSimplify (True)/ExprAF2
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.2680841773468181e-5 | 1.2721902994944566e-5 | 1.2775538248616097e-5 |
| Standard deviation | 1.035668540428624e-7 | 1.2247803675726367e-7 | 1.4886991682807457e-7 |
Outlying measurements have no (7.091836734693845e-3%) effect on estimated standard deviation.
mathSimplify (True)/ExprAF3
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.260408784949827e-5 | 2.2659778501127175e-5 | 2.271071749348347e-5 |
| Standard deviation | 1.2007798132330264e-7 | 1.4389357092329197e-7 | 1.93925157260855e-7 |
Outlying measurements have no (7.691845441980644e-3%) effect on estimated standard deviation.
mathSimplify (True)/ExprAF4
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.47181391358866e-5 | 6.484049289480494e-5 | 6.495929588059714e-5 |
| Standard deviation | 1.799878025759811e-7 | 2.7531473892038095e-7 | 3.6673883968098335e-7 |
Outlying measurements have no (9.258450519696041e-3%) effect on estimated standard deviation.
mathSimplify (True)/ExprAF5
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 9.638298260424497e-5 | 9.656651668205712e-5 | 9.673409117525655e-5 |
| Standard deviation | 4.379166037421077e-7 | 5.187889265034461e-7 | 6.092754578232649e-7 |
Outlying measurements have no (9.998979695949463e-3%) effect on estimated standard deviation.
mathSimplify (True)/ExprAF6
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.4648859518339734e-4 | 1.4701977341904257e-4 | 1.4756659156260371e-4 |
| Standard deviation | 6.687912564329716e-7 | 1.3241897951513645e-6 | 1.981225873873553e-6 |
Outlying measurements have slight (1.0868252626494367e-2%) effect on estimated standard deviation.
mathSimplify (True)/ExprAF7
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.1444906419982676e-4 | 2.1512393607697438e-4 | 2.1594666114339812e-4 |
| Standard deviation | 1.4135430605838384e-6 | 1.899966211851582e-6 | 2.708287680026875e-6 |
Outlying measurements have slight (1.1903033822035055e-2%) effect on estimated standard deviation.
mathSimplify (True)/ExprAF8
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.5752361364029e-4 | 4.5831108870699676e-4 | 4.598377844109847e-4 |
| Standard deviation | 1.8705115261146307e-6 | 2.7650305337320773e-6 | 3.984178807564517e-6 |
Outlying measurements have slight (1.4489619377162505e-2%) effect on estimated standard deviation.
mathExpand (False)/ExprAF2
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.849028116628718e-7 | 5.913183232508995e-7 | 5.989580373666238e-7 |
| Standard deviation | 1.4339238995124151e-8 | 1.9426866215046554e-8 | 2.4562527485710766e-8 |
Outlying measurements have moderate (0.44776357779740095%) effect on estimated standard deviation.
mathExpand (False)/ExprAF3
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.9409537105105532e-6 | 2.953132143268301e-6 | 2.966807297148069e-6 |
| Standard deviation | 3.274145582000816e-8 | 3.9102930740578515e-8 | 4.745968234933246e-8 |
Outlying measurements have moderate (0.10000522133373703%) effect on estimated standard deviation.
mathExpand (False)/ExprAF4
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.463538477827541e-6 | 6.488939184874514e-6 | 6.512627933478711e-6 |
| Standard deviation | 6.227720392767434e-8 | 7.406141108195991e-8 | 9.694275959587533e-8 |
Outlying measurements have slight (6.745476528400737e-2%) effect on estimated standard deviation.
mathExpand (False)/ExprAF5
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.4051611206473159e-5 | 1.4106270483065728e-5 | 1.4173478284860128e-5 |
| Standard deviation | 1.2341415428805134e-7 | 1.6901017112603748e-7 | 2.257913159455597e-7 |
Outlying measurements have slight (6.832204084180898e-2%) effect on estimated standard deviation.
mathExpand (False)/ExprAF6
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.0508946653976528e-5 | 3.064526001675746e-5 | 3.0833673003240696e-5 |
| Standard deviation | 3.289069415583802e-7 | 4.3272165980219e-7 | 5.783836615678798e-7 |
Outlying measurements have slight (8.494195379188159e-2%) effect on estimated standard deviation.
mathExpand (False)/ExprAF7
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.771984511962052e-5 | 6.804750712479109e-5 | 6.82916998859923e-5 |
| Standard deviation | 6.536330622019536e-7 | 9.312548575536538e-7 | 1.198599471490802e-6 |
Outlying measurements have slight (7.098868776368582e-2%) effect on estimated standard deviation.
mathExpand (False)/ExprAF8
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.476949888188083e-4 | 1.4851599583893749e-4 | 1.494341461522868e-4 |
| Standard deviation | 1.916040654229894e-6 | 2.261731772925174e-6 | 2.7853108763881582e-6 |
Outlying measurements have slight (7.324929372323677e-2%) effect on estimated standard deviation.
mathExpand (True)/ExprAF2
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.640274961152663e-7 | 6.755079366187608e-7 | 6.911558969379319e-7 |
| Standard deviation | 2.7226028078299693e-8 | 3.584868975165528e-8 | 4.9680271141624083e-8 |
Outlying measurements have severe (0.6783422319995227%) effect on estimated standard deviation.
mathExpand (True)/ExprAF3
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.2628371877321188e-6 | 3.2804944141523317e-6 | 3.296244396990926e-6 |
| Standard deviation | 3.497727016969132e-8 | 4.192643442159338e-8 | 5.53381769702932e-8 |
Outlying measurements have slight (9.007351663753507e-2%) effect on estimated standard deviation.
mathExpand (True)/ExprAF4
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 7.17370977618308e-6 | 7.225063897749037e-6 | 7.2566124544658704e-6 |
| Standard deviation | 8.61198534114974e-8 | 1.0642003840558806e-7 | 1.3583615524936203e-7 |
Outlying measurements have moderate (0.10591525893570194%) effect on estimated standard deviation.
mathExpand (True)/ExprAF5
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.6451395025242327e-5 | 1.6501608974003745e-5 | 1.6550794494059164e-5 |
| Standard deviation | 1.300366709732361e-7 | 1.4567078600989864e-7 | 1.838694110533557e-7 |
Outlying measurements have no (7.352537722908112e-3%) effect on estimated standard deviation.
mathExpand (True)/ExprAF6
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.489847733329877e-5 | 3.5019289713510944e-5 | 3.5235485850235075e-5 |
| Standard deviation | 3.0328355636845783e-7 | 3.846336270787168e-7 | 5.226490463925071e-7 |
Outlying measurements have no (8.263888888888968e-3%) effect on estimated standard deviation.
mathExpand (True)/ExprAF7
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 7.806534538843951e-5 | 7.920367903581012e-5 | 8.057183390880403e-5 |
| Standard deviation | 2.7932130098930644e-6 | 3.1832218982220415e-6 | 4.0482861714875324e-6 |
Outlying measurements have moderate (0.3831772239376825%) effect on estimated standard deviation.
mathExpand (True)/ExprAF8
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.622612883962656e-4 | 1.6298334287449332e-4 | 1.6384848392351676e-4 |
| Standard deviation | 1.3883603455285938e-6 | 1.954691553527788e-6 | 2.595239914269663e-6 |
Outlying measurements have slight (1.1234504132231342e-2%) effect on estimated standard deviation.
understanding this report
In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.
- The chart on the left is a kernel density estimate (also known as a KDE) of time measurements. This graphs the probability of any given time measurement occurring. A spike indicates that a measurement of a particular time occurred; its height indicates how often that measurement was repeated.
- The chart on the right is the raw data from which the kernel density estimate is built. The x axis indicates the number of loop iterations, while the y axis shows measured execution time for the given number of loop iterations. The line behind the values is the linear regression prediction of execution time for a given number of iterations. Ideally, all measurements will be on (or very near) this line.
Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.
- OLS regression indicates the time estimated for a single loop iteration using an ordinary least-squares regression model. This number is more accurate than the mean estimate below it, as it more effectively eliminates measurement overhead and other constant factors.
- R² goodness-of-fit is a measure of how accurately the linear regression model fits the observed measurements. If the measurements are not too noisy, R² should lie between 0.99 and 1, indicating an excellent fit. If the number is below 0.99, something is confounding the accuracy of the linear model.
- Mean execution time and standard deviation are statistics calculated from execution time divided by number of iterations.
We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)
A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.